The summary presented below was prepared by the USDA Food and 
Nutrition Service based on two projects being completed by 
Mathematica Policy Research, Inc. Formal written reports are not yet 
available; this summary is based on data provided by the contractor and on 
draft reports available as of September 2003. We do not expect the 
findings in this summary to change in any significant way, but more detailed 
information will be available in the near future. 



NSLP Certification Accuracy Research 
Summary of Preliminary Findings 

Prepared by the Food and Nutrition Service, USDA 

The National School Lunch Program (NSLP) and School Breakfast Program (SBP) serve 
about 3.9 billion free and reduced-price meals to children annually. The food consumed at these 
meals makes up an important component of these children's overall nutritional intake. In recent 
years, concerns have grown about the integrity of the program's system for establishing 
eligibility for free or reduced-price meals. Several data sources suggest that the number of 
children approved for free or reduced-price meals from families with incomes too high to qualify 
for the benefits they receive is large and perhaps growing. 

This document summarizes the preliminary findings of two projects being completed by 
Mathematica Policy Research, Inc. on the subject of certification accuracy in the National School 
Lunch Program: Evaluation of the National School Lunch Program Application/Verification 
Pilot Projects: Findings on Deterrence, Barriers and Accuracy , and Case Study of Verification 
Outcomes in Large Metropolitan Areas. 

Evaluation of the National School Lunch Program Application/Verification Pilot Projects: 

Findings on Deterrence, Barriers and Accuracy 

The U.S. Department of Agriculture (USDA) sponsored the NSLP Application/Verification 
Pilot Projects to test ways to improve the process for certifying students for free or reduced-price 
meals. This report presents some preliminary impact findings from these pilot projects, which 
examined two alternatives to the current application-based certification process — Up-Front 
Documentation and Graduated Verification — in twelve public school districts. The pilot projects 
test certification procedures that are substantively different from the Administration's 
certification accuracy recommendations. A summary of the range of program improvements 
recommended by the Administration to address NSLP integrity issues as part of a balanced 
approach to Child Nutrition reauthorization appears at the end of this report. 
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A descriptive analysis of the first-year of pilot project operations was released in 2002. 
A broader evaluation of these projects, now being finalized, explores a wide range of 
potential program impacts resulting from the pilot. This document focuses on the impact of 
pilot procedures on three key measures of program performance: 

• The rate of certification among ineligible households 

• The rate of certification among eligible households 

• Certification accuracy - the proportion of certifications that are correct 

In launching the pilots, USDA was seeking to identify changes to the certification process 
that would deter certification among ineligible households without causing barriers to 
certification among eligible households. The end goal was an overall improvement in 
certification accuracy as compared to normal certification procedures. Each pilot procedure 
was evaluated in terms of these measures. 

Current Certification Procedures 

Under federal guidelines, children living in families with incomes of 130 percent or less of 
the federal poverty level qualify for free meals, while those in families with income of between 
130 and 185 percent of poverty qualify for reduced-price meals. In addition, children receiving 
Food Stamp, Temporary Assistance for Needy Families (TANF), or Food Distribution Program 
on Indian Reservations (FDPIR) benefits are categorically eligible for free meals. 

Most families that become certified for free meals do so by submitting an application on 
which they report their income and household size or Food Stamp/TANF/FDPIR case number. 
Under standard federal rules, no documentation of household income or benefit receipt is 
required at the time of application. In the verification process, districts select a small sample of 
applications that have already been approved and collect income or benefit documentation from 
approved families in order to verify their eligibility for free or reduced-price meals. 

Pilot Procedures 

Nine districts included in the evaluation implemented Up-Front Documentation. Under this 
pilot, districts required all applicants for free or reduced-price meals to provide documentation, 
either of their income or participation in Food Stamps/TANF/FDPIR, with the application. 

Three districts included in the evaluation implemented Graduated Verification, under which 
the standard verification process currently practiced by schools was expanded incrementally. 
Districts using Graduated Verification first verified a standard sample, but also conducted 
additional rounds of verification using increasing samples up to 100 percent if at least 25 percent 
of verified applications had benefits reduced or terminated in the prior round. 



U.S. Department of Agriculture, Food and Nutrition Service, Office of Analysis, Nutrition and Evaluation. 
National School Lunch Program Application / Verification Pilot Project: Report on First Year Experience (CN-02- 
AV). Alexandria, VA: 2002. 
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To estimate the impacts of these procedures, the pilot evaluation compared average 
outcomes among a sample of households in pilot districts with those in comparison districts with 
similar characteristics. 

Approximately 3,000 households with children enrolled in the 12 pilot and 12 comparison 
districts were interviewed in this study. The sample included approximately 1,300 approved for 
free or reduced price meals while approximately 1,700 were not approved for either free or 
reduced price meals in October 2002. 

Administrative Data vs. Evaluation Results 

USDA's descriptive analysis of administrative data from the first year of pilot 
implementation showed that reduced rates of certification and participation in the free and 
reduced price categories followed implementation of Up-Front Documentation and 
Graduated Verification, with increases in participation at the full price level, resulting in 
modest reductions in overall participation rates^. 

Unlike the administrative data used in the first year analysis, the evaluation provides 
information on the underlying eligibility status of the households in the pilot and comparison 
districts in the third year of pilot implementation. This data can be used to assess the 
cumulative impact of the pilot procedures on: 

• deterrence (i.e., lower certification rates among ineligible households); 

• barriers (i.e., lower certification rates among eligible households); and 

• changes in certification accuracy (the proportion of certifications that are correct) 

as compared to standard certification procedures. Note that certifying those not eligible for 
any benefit (neither for free or reduced price meals) results in higher erroneous payments 
than certifying a reduced-price-eligible student for free meals. 

Evaluation Findings 

Certification Accuracy; In comparison districts operating under standard certification 
procedures, roughly one in five free certifications are of ineligible households. The study 
found that, in these districts: 

• Among all students certified for free meals, 18.4 percent were ineligible for free meals. 

• Among those children certified but ineligible for free meals, 69.7 percent were eligible for 
reduced price meals and 30.3 percent were not eligible for either free or reduced price meals. 

• Examining the broader measure of certification accuracy - the proportion of those certified 
for either free or reduced price meals that are not eligible for either free or reduced price 



U.S. Department of Agriculture, Food and Nutrition Service, Office of Analysis, Nutrition and Evaluation. 
National School Lunch Program Application / Verification Pilot Project: Report on First Year Experience (CN-02- 
AV). Alexandria, VA: 2002, pp. i-ii. 
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benefits - the rate of certification inaccuracy was 1 1.6 percent in the average comparison 
district. 

Eligible Non-Certified Children: Some students eligible for benefits were not certified. 

• In Up-Front Documentation comparison districts, 58.7 percent of all children eligible for free 
meals were certified for free meals while 7.7 percent were approved for reduced price meal 
benefits; 33.6 percent of free ehgible children were not approved for either free or reduced 
price meal benefits. 

• In Graduated Verification comparison districts, 79.2 percent of all children eligible for free 
meals were certified for free meals while 7.8 percent were approved for reduced price meal 
benefits; 13.0 percent of free ehgible children were not approved for either free or reduced 
price meal benefits. 

Pilot Impacts on Deterrence, Barrier and Accuracy^ 

Neither Up-Front Documentation nor Graduated Verification resulted in observable 
deterrence of erroneous certifications. 

• For Up-Front Documentation, 3.3 percent of students ineligible for free meals in pilot 
districts were certified for these benefits, while 4.1 percent of free-ineligible students were 
certified in comparison districts. This difference, while moderately large in percentage 
terms, is not statistically significant. 

• For Graduated Verification, no differences were found in the proportions of ineligible 
students erroneously certified for free meals in pilot and comparison districts. 

Both interventions caused barriers to certification among some eligible students. 

All of the measures of barrier tested showed lower certification rates in pilot districts; some of 
these differences were statistically significant. For example: 

• Up-Front Documentation 

o The percentage of free-eligible students certified for free or reduced-price meals in pilot 

districts was 53.1 percent, versus 62.4 percent in comparison districts. This difference is 

statistically significant, 
o The difference between the corresponding findings for free certifications of free-eligible 

students, 47.6 percent for pilot districts and 53.7 percent for comparison districts, is not 

statistically significant. 

• Graduated Verification 

o The percentage of free-eligible students certified for free meals in pilot districts was 56.0 
percent, versus 69.3 percent in comparison districts. This difference is statistically 
significant. 

^ All statistics that compare pilot and comparison districts for the remainder of this document control for 
differences in household and district characteristics and exclude directly certified children. 
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o The difference between the corresponding findings for free or reduced-price certification 
of free-eligible students, 64.2 percent in pilot districts and 79.9 percent in comparison 
districts, is statistically significant. 

Compared to current procedures, neither intervention changed certification accuracy at a 
statistically significant level. 

The full set of final evaluation reports will explain the findings above in more detail, and 
provide further information on the impacts of the interventions on participation levels, 
administrative burden for school food authorities (SFAs), the fidelity with which pilot districts 
implemented the demonstrated procedures, and other aspects of the project. 

Case Study of National School Lunch Program Verification Outcomes In Large 

Metropolitan Areas 

Because the pilot evaluation did not include large school districts, FNS sought to learn about 
certification accuracy in large school districts through a study in 21 SFAs, located in 7 large 
metropolitan areas (Boston, Orlando, Virginia Beach-Norfolk-Newport News, Los Angeles, Salt 
Lake City, Tulsa, and Minneapolis-St. Paul). A fiill report presenting the results is forthcoming. 

Objectives and Methodology 

The objectives were to assess the outcomes of the verification process in study SFAs, and to 
independently determine the income eligibility category of two groups of households: 1) a 
sample of households that did not respond to verification (and are therefore subject to losing 
eligibility for benefits), and 2) a sample of households whose benefit levels were unchanged in 
the verification process. Determining the eligibility status of each of these groups provides a 
useful measure of the effectiveness of verification in detecting and reducing inaccuracy among 
sampled certifications. 

The study recorded the verification outcome of all cases selected for verification in Fall 
2002 in the study SFAs, and conducted in-home interviews with samples of non-respondent 
households and households whose benefits were unchanged. 

The study also collected information on the rate of reapproval among households initially 
approved on the basis of income and household size who did not respond to verification. 

Findings 

Among households selected for verification, about one-half did not respond. On average: 

• 50 percent of households in the verification sample responded to the request for verification 
by the December 15, 2002 deadline, while 50 percent did not. 

• Among those that responded: 

o 65 percent had benefits unchanged; 

o 32 percent had benefits reduced or terminated; and 

o 3 percent had benefits increased. 
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Among non-respondent households, about one-half were eligible for at least the benefit they 
had been receiving prior to verification. 

• Among non-respondents that had been approved for free meals: 

o 51 percent were eligible for free meals; 

o 26 percent were eligible for reduced price meals; and 

o 23 percent were not eligible for either. 

• Among non-respondents that had been approved for reduced price meals 

o 23 percent were eligible for reduced price meals; 
o 31 percent were eligible for free meals; and 
o 46 percent were not eligible for either. 

• Among all non-respondents who remained enrolled in the school districts, 25 percent re- 
applied for meal benefits between December 16, 2002 and March 1, 2003, and were re- 
approved. However, of those that were reapproved: 

o 22 percent of those reapproved were not eligible for the benefits for which they had 
been reapproved. 

o An additional 14 percent were eligible for a higher level of benefits than those for 
which they had been reapproved. 

Among households whose benefits were unchanged in the verification process, about one- 
third were ineligible for their approved level of benefits two to three months after completing 
the verification process. 

• Among free approved children whose eligibility had been unchanged in the verification 
process, the study found that as of February/March 2003: 

o 64 percent were eligible for free meal benefits; 

o 27 percent were not eligible for free meal benefits, but were eligible for reduced price 

meal benefits; and 
o 9 percent were not eligible for either. 

• Among reduced price approved children whose eligibility had been unchanged in the 
verification process, the study found that as of February/March 2003: 

o 42 percent were ehgible for reduced price meal benefits; 
o 25 percent were eligible for free meal benefits; and 
o 33 percent were not eligible for either. 

• It should be noted that the two measures of household circumstances were made at different 
times, so that changes in circumstance over the intervening 2-3 months may contribute to 
differences between the two. 

USDA Conclusions: Implications for Policy 

While not based on nationally-representative data, these findings, coupled with data from 
national surveys and audits, have informed USDA' s judgment that: 

• Certification of ineligible children is a problem. 

• At the same time, there are a number of low-income children not approved for free 
meals. 
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• Neither documentation of eligibility at application, nor graduated verification triggered 
by problems found in an initial sample, as conducted in the pilot sites, had desirable 
effects on certification accuracy. 

• The current application and verification process requires improvement. Both changes 
in policy and administrative practice are warranted. 

The Administration has recommended a balanced approach to Child Nutrition reauthorization, 
which includes a range of program improvements to safeguard access while addressing NSLP 
integrity issues. These proposals are substantively different than up-front documentation or 
graduated verification. We have identified the following actions to improve integrity, as well as 
access: 

• Require direct certification for free meals through the Food Stamp Program, to improve 
certification accuracy over paper applications while increasing access for the lowest-income 
families and reducing the application and verification burden for families and schools. 

• Permit households to submit a single application covering all children attending school, 
and provide for yearlong certifications. These improvements reduce certification and 
verification burden while reducing potential for error. 

• Enhance verification of paper-based applications by drawing verification samples early in 
the school year, expanding the verification sample; and including both a random sample and 
one focused on error-prone applications in each school. 

• Minimize barriers for eligible children who wish to remain in the program by requiring 
a robust, consistent effort in every State to follow-up with those who do not respond to 
verification requests, including a minimum of three contacts in writing and by phone. 

Other protections to minimize barriers for eligible children being discussed include: 

• allowing the use of Medicaid certification for verification purposes; 

• exempting certain vulnerable populations (such as homeless or migrant children) from 
the verification sample; and 

• allowing parents to identify on applications a third party that can assist them in 
responding to a verification request. 

• Initiate a series of comprehensive demonstration projects to test alternative 
mechanisms for certifying and verifying applicant information, including use of wage 
data matching that identifies eligible and ineligible households and a nationally 
representative study of overcertification error and the number of program dollars lost to 
program error. 
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